Fisher Information in Gaussian Graphical Models

This note summarizes various derivations, formulas and computational algorithms relevant to the Fisher information matrix of Gaussian graphical models with respect to either an exponential parameterization (related to the information form) or the corresponding moment parameterization. 1 Gauss-Markov Models The probability density of a Gaussian random vector x ∈ R may be expressed in information form: p(x) ∝ exp{− 1 2 xJx+ hx} with parameters h ∈ R and J ∈ R, where J is a symmetric positive-definite matrix. The mean and covariance of x are given by: μ , p[x] = Jh P , p[(x− μ) (x− μ)] = J where we denote expectation of f(x) with respect to p by p[f ] , ∫ p(x)f(x)dx. In this note, we focus on the zero-mean case μ = 0. Thus, h = 0 as well. A Gaussian distribution is Markov on a graph G = (V,E), with vertices V = {1, . . . , n}, if and only if Jij = 0 for all {i, j} 6∈ E. Thus, Markov models have a reduced information parameterization based on a sparse J matrix. 2 Exponential Family and Fisher Information The Gaussian distributions can also be represented as an exponential family : p(x) = exp{θφ(x)− Φ(θ)} with parameters θ ∈ R and sufficient statistics φ : R → R. To obtain the information form we define statistics: φ(x) = (xi , i ∈ V ) ∪ (xixj , {i, j} ∈ E) The corresponding exponential parameters then correspond to elements of the information matrix: θ = (− 1 2 Jii, i ∈ V ) ∪ (−Jij , {i, j} ∈ E)